Digital overlay

ABSTRACT

Systems and methods for using a digital overlay screen during a video telecommunication. The digital overlay screen comprises content, such as notes or a script, which a presenter may reference during said video telecommunication. The digital overlay screen may tether to an underlying network conferencing program or a video captured from an image capturing device of a user&#39;s device. The digital overlay screen is arranged relative to the image capturing device such that the presenter can review the content therein while simultaneously establishing and maintaining a line of sight with the device. The digital overlay screen may overlay on a video feed of the telecommunications without distorting said feed, and without being transmitted or visible to third party recipients of the presentation.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional PatentApplication No. 63/156,277, filed Mar. 3, 2021, the entire contents anddisclosures of which are hereby incorporated by reference.

FIELD

The subject matter described herein relates generally to the field ofvideo telecommunications and, in particular, to systems and methods forthe use of digital overlays in the recording, streaming and presentationof content over video teleconferencing services.

BACKGROUND

As digital electronics and network communications are becomingincreasingly popular, users are shifting to real-time interactivenetworked conferences as a means to facilitate business meetings,speeches or presentations to groups, medical & teletherapy meetings,lectures to students, lawyer arguments to court, and the like. Networkconferencing platforms, including but not limited to Zoom, MicrosoftTeams, BlueJeans, FaceTime, Skype, Webex Meetings, and GoTo Meeting,allow individuals to share text, images, audios, videos, and other databy sharing their computer screen in real-time to participants in remotelocations. In many instances, a user engaged in a video-basedinteraction may have content prepared in anticipation thereof. Forexample, a presenter may have a script, notes, talking points, key data,information, or the like, that they may wish to review or referenceduring a video presentation. However, these are not shared with theaudience or even desired to be shared with the audience by the presenterand so instead when a presenter viewing, referring to or reading fromnotes positioned outside the video stream's device's (e.g., the camera)field of view he or she may fail to establish and maintain a line ofsight with said video streaming device, thus, leading to the presenterlosing “eye contact” with the audience and resulting in a distracting,less professional and/or persuasive, and lower quality viewingexperience for the viewer(s) and less effective presentation,discussion, meeting, argument, sales pitch, conference, etc. by thepresenter. And a presenter's associates may wish in the moment to tellthe presenter to say or not say something in particular and be able tocommunicate to the presenter without the presenter looking away and/ornotifying the viewers/audience that the presenter is being givendirections, information or data by third parties, but currently thatrequires looking away to another device or screen or being handed a notethat one then has to look away and read. Therefore, it would beadvantageous for content (pre-determined or real-time delivered bypresenter's associates) the presenter wishes to reference during apresentation to be digitally overlaid via a display in a manner suchthat said content is visible to the presenter (e.g., a subject of thevideo), and not visible to a recipient or audience of the video stream,or optionally, not rendered as part of the video recording. It wouldalternatively and additionally be further advantageous for the digitaloverlay to be tethered to the camera display of whatever networkconferencing platform or program is being used, either as a digitaloverlay within the platform or program or as a stand-alone program wherethe tethering is done by the user or the stand-alone program, so thatthe camera-default field of view has the digital overlay field over it.It would also be advantageous to have these pre-determined data fieldsand message communications to occur when one is using augmented reality(AR) or virtual reality (VR) glasses, smart glasses or smart contactlenses to video present in lieu of a traditional computer/camera toengage in video conferencing activities or when using AR or VR glasses,smart glasses or smart contact lenses to video present in conjunctionwith a traditional computer/camera. Still further, it would beadvantageous when a user is using AR or VR glasses, smart glasses orsmart contact lenses in a live interaction and speaking to a physicallypresent audience (such as a lawyer to a judge or jury, or a speaker in aTED talk, a doctor to a patient, a sales pitch to a client, for example)to have access to the same data fields or message communicationsfeatures.

Additionally, the presenter and subject of the video may not havesufficient experience or knowledge relating to the substance of thepresented dialogue, or may need help or assistance in theirpresentation, and may wish to consult or confer with a colleague, or maywish or be required to receive advice and guidance from a colleague,during the video (or live) presentation in a manner that is notdistracting and in a manner that the viewers do not even know isoccurring. Thus, it would also be advantageous to transmit or retrievemessages, share notes, files, images, and/or other data by and betweenthe speaker and an associate in real-time, wherein said data orcommunications are digitally overlaid and broadcasted on the display ina manner such that said outgoing/incoming information the speaker mayutilize in making their presentation is visible only to them, and notvisible to a recipient of the video stream, and optionally, not renderedas part of the video recording.

It would be further advantageous to have the aforementioned digitaloverlays be oriented such that the rendered video stream gives theappearance to the viewer that a subject has not lost eye contact withthe audience and is looking directly at the camera for the duration ofthe video presentation so that they present more appropriately,professionally, polished and/or in a more appealing manner as a speakeror presenter.

SUMMARY

The field of the subject matter described herein relates generally tovideo telecommunications. In particular, the field relates to systemsand methods for the use of digital overlays in the recording, streamingand presentation of content over video conferencing services.

In an embodiment, an event-based digital overlay system includes anapplication, wherein text-based and interactive content is displayedover the native video streaming field of view for the video conference(i.e., over the Zoom or FaceTime (or other such program) field of view)therein for a presenter to reference during a video telecommunication.Digital imagery data of the video may be captured from, for example, animage capturing device of a user's device, or an external cameraoperatively coupled to a user's device, and may be streaming to and froma network conferencing program or the like. The digital overlay systemallows users to create digital overlay events comprising digital overlayscreens with fields having content the user wishes to reference orreview in real-time during a presentation. The digital overlay screencomprising said content may be tethered to the camera display of theunderlying network conferencing program or may be a separate programthat is then opened and sized to the video streaming screen visible tothe subject. Further, the video feed from the image capturing device ornetwork conferencing program, or the like, are simultaneously displayedon an electronic display of the user's device via a user interface. Thedigital overlay screen is projected on the user interface so as to atleast partially (if not also fully) superimpose or overlay over thevideo feed in a manner in which the video imagery data is notsignificantly blocked or distorted. The digital overlay screen is alsomethodically arranged such that the presenter may read the contenttherein without seeming to look away from the camera's focal point andalso without losing sight of the viewers or having their appearancesdistorted or lost. The digital overlay screen and its content is onlyvisible to the presenter, and not visible to the recipients of thepresentation. The content may be customizable as to location on thedigital overlay screen or may have pre-determined locations. In thisway, the user is able to discreetly review content from the screen'sfields while concurrently viewing the video feed, all while appearing toand/or in fact maintain eye contact with third parties interacting viathe video feed or a network conferencing program.

Other systems, methods, features and advantages of the subject matterdescribed herein will be or will become apparent to one with skill inthe art upon examination of the following figures and detaileddescription. It is intended that all such additional systems, methods,features and advantages be included within this description, be withinthe scope of the subject matter, and be protected by the accompanyingclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description, isbetter understood when read in conjunction with the accompanyingdrawings. The accompanying drawings, which are incorporated herein andform part of the specification, illustrate a plurality of embodimentsand, together with the description, further serve to explain theprinciples involved and to enable a person skilled in the relevantart(s) to make and use the disclosed technologies.

FIG. 1 is a block diagram illustrating a digital overlay system,according to an embodiment.

FIG. 2 is a flow chart illustrating the ecosystem of the digital overlayapplication in the digital overlay system, according to an embodiment.

FIG. 3 is a flow chart illustrating an example method for creatingdigital overlay events with the digital overlay system, according to anembodiment.

FIG. 4 is an exemplary diagram of a multi-participant videotelecommunications interaction according to an embodiment.

FIG. 5 is an exemplary user interface, illustrating event controls,according to an embodiment.

FIG. 6 is an exemplary process for the accessibility controls of anevent, according o an embodiment.

FIG. 7 is an exemplary user device comprising an exemplary userinterface, according to an embodiment.

FIG. 8 is an exemplary user interface, according to an embodiment.

FIG. 9 is a close-up of the exemplary user interface illustrated in FIG.8 wherein a field comprising content has been enlarged, according to anembodiment.

FIG. 10 is an exemplary user device comprising an exemplary userinterface, according to an embodiment.

DETAILED DESCRIPTION

While the present subject matter described herein is susceptible ofembodiment in various forms, some of which as shown in the drawings,hereinafter will be described the presently preferred embodiments of thesubject matter with the understanding that the present disclosure is tobe considered as an exemplification of the subject matter, and it is notintended to limit the subject matter to the specific embodimentsillustrated.

The subject matter described herein addresses the aforementioneddisadvantages by providing improved systems and methods for renderingvideo recordings and presentations with digital overlays. Specifically,an event-based digital overlay system is described which provides adigital overlay application for dynamically displaying text-basedcontent, in real-time, during a presentation, recording, and/orstreaming of a video. FIG. 1 includes a block diagram illustrating theevent-based digital overlay system 104. In FIG. 1, 101 represents thedigital overlay application which may be stored and executed by a userdevice 110, such as a computer, and uses software which executes programcode to interface with an optically active device or image capturingdevice 103 of said user device 110, which may be communicatively coupledto a video browser program, video rendering program, or a networkconferencing platform 100 of the like.

Specifically, the user device 110 may comprise at least one processor109, an optically active device or image capturing device 103, a displaygenerating unit 102, and at least one memory 111. The at least oneprocessor 109 is capable of manipulating data in accordance with a setof instructions 105. For example, the at least one processor 109 may beconfigured to execute programs based on software or a set ofinstructions 105 from the memory 111. For instance, the at least oneprocessor 109 is configured to initiate communication with theunderlying network conferencing platform 100 and/or the digital overlayapplication 101 to provide data to and/or request data from saidunderlying program 100 and/or the digital overlay application 101. Forinstance, the digital overlay application 101 can initiate a request tothe underlying program to stream a video recording, join a live video,and communicate with third parties logged onto the underlyingvideo/network conferencing program 100, etc. Additionally, in responseto a set of instructions, the digital overlay application 101 maygenerate a digital overlay screen 112 to be displayed on a userinterface 115. In some embodiments, a bus architecture is utilized. Thebus may include any number of interconnecting buses and bridgesdepending on the digital overlay application 101, the underlying program100, and the overall design constraints.

The digital overlay application 101 interfaces with one of theunderlying network conferencing programs 100 and cooperates with thedisplay generating unit 102 so as to generate a digital overlay screen112, which may tether to the camera display of the user device orunderlying program 100 and is electronically projected onto the user'sdevice 110 via a user interface 115. Concurrently, video captured fromthe image capturing device 103 of the user device 110 is transmitted orstreamed to a video rendering of one or more remote devices 150, withouttransmission of the digital overlay screen 112. The digital overlaysystem 104 may capture the digital imagery, or video feed 120, capturedby said image capturing device 103 and stream or transmit it to the userdevice 110, and a remote server or remote devices 150 connected to theunderlying networking program 100. In some embodiments, the video feedmay comprise digital imagery of the user, other audience members fromremote devices (e.g., zoom participants), or a combination of both. Thedigital overlay screen 112 may overlay the video feed 120 such that itdoes not distort the digital imagery data transmitted therefrom.

The network conferencing program 100 may be Zoom, Microsoft Teams,BlueJeans, FaceTime, Skype, Webex Meetings, GoTo Meeting, or any otherprogram which allows individuals to video conference efficiently and inreal-time to participants on remote devices in remote locations.Optically active devices or image capturing devices 103 may include acamera device or recording device of a personal computer, portable,mobile, wearable, hand-held device (e.g., smartphones, laptops, orpersonal digital assistants (PDAs)), or other machines of the like. Morespecifically, the image capturing or recording device 103 may be theintegrated front facing camera or back facing camera of a user device,or an external camera operatively coupled to a user device. Moreover,optically active devices 103 may include AR, VR, or smart glasses, smartlenses and similar gadgets, such as AR/VR Glasses.

The optically active device 103 may be communicatively coupled via anetwork 1050, program, or the like, that is being executed on a videorendering program 100 on one or more remote devices 150. For example,the camera or image capturing device 103 may be communicatively coupledwith a network, such as the internet, intranets, extranets, wide areanetworks (WANs), local area networks (LANs), wireless networks, wirednetworks, a plurality of the aforementioned networks, and other suitablenetworks determined by one skilled in the art.

The digital overlay system 104 also enables network conferencingprograms 100, and the like, to embed the functionality of theapplication 101 to their program100 by adding a script to theirplatform. In some embodiments, the script may be a single line of Javascript. The Java script may load from one or more servers that containthe instructions required to interface or systematically tether thedigital overlay application 101 to said program 100.

General Overview of the Digital Overlay Application Ecosystem

FIG. 2 illustrates the general ecosystem of the digital overlayapplication 101. The digital overlay system 104 comprises the digitaloverlay application 101, wherein a user (or presenter) may create anevent 200 associated with a video feed 120 for a presentation,recording, or streaming. An event 200 may include one or more uniquedigital overlay screens 112 comprising of one or more fields 220 whichinclude uploaded, downloaded, transmitted, inputted, or otherwisegenerated digital content 240, such as messages, notes, a script, or thelike, which are associated with a video presentation. As detailedfurther below, the digital overlay screen 112 may include a chatinterface known in the art that enables approved participating viewersor other third-party attendees at remote terminals of the digitaloverlay application to communicate with each other and the presenter viamessage exchange in real-time throughout an event. In this way, a newcontent field or box 220, comprising messages, audio, pictures, files,multimedia, and other data and information of the like, may betransmitted to the presenter during the event 200.

As shown in FIG. 1, the digital overlay screen 112 comprising thedigital content 240 (including content 240 transmitted from externalsources) is then digitally overlaid over the presentation's video feed120 on the user interface 115. In this manner, the content 240 projectedfrom the digital overlay screen 112 is visible to the presenter (and anyinvited associates who are helping or communicating with the presenterby pre-approval), but not visible to the audience, and is not distortingthe underlying video feed 120. Moreover, the digital overlay screen 112may comprise AR or VR functionality such that the projected digitaloverlay screen 112 is a digital AR or VR overlay screen.

Specifically, the user creates an event on the digital overlayapplication 101 which corresponds to one or more video presentations.Once an event 200 is created, the user may input image, text-based, orinteractive content 240 for the event 200. Said digital content 240 isthen uploaded into one or more fields 220, and may then be locallyrecorded, for example, within a storage module of the digital overlayapplication 101 or stored in an event-files database, such as anevent-storing database. The content 240 may be generated on a digitaloverlay screen 112 when the user opens the event 200. Specifically, thedigital overlay screen 112 is dynamically displayed on the userinterface 115 and presented on an electronic display of the user'sdevice 110 when the user opens the event 200. More specifically, theuser interface 115 comprises the digital content 240 from the digitaloverlay screen 112 in addition to the video feed 120 captured from theimage capturing device 103 or associated with the underlying networkconferencing program 100. Even more specifically, the digital overlayscreen 112 is projected onto the user interface 115 such that it atleast partially superimposes or overlays the video feed 120. The digitaloverlay screen is configured over the video feed in a manner such thatit does not distort the video feed's 120 imagery data or alert thirdparty video recipients of its existence or use.

Digital Overlay Application

FIG. 3 is a flow chart illustrating an example method 300 for creatingdigital overlay events 200 with the digital overlay system 104. Thedigital overlay application 101 is designed to provide users with theability to create personalized user profiles distinguishable by uniqueuser log-in credentials. In block 301, the user launches or starts thedigital overlay application 101. Once the digital overlay application101 is launched or started, in block 302, the user may log into theiruser profile and access the digital overlay application's 101event-selection interface. In block 303, the user creates one or moredigital overlay events 200 through the event-selection interface. Inblock 304, the user saves the digital overlay event(s) 200 to theevent-storing database. In block 305, the saved digital overlay events200 are then associated with the user and their respective presentationvia the user's profile, and may be accessed by the user at a later time.

Turning to FIG. 4, users can install the digital overlay application 101on their user device 110, such as a personal computing device (or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by the processing systemdescribed herein) to establish an event 200 through the digital overlayapplication 101 which may operate as a standalone application (a digitaloverlay platform), or interface with a network conferencing platform100. Specifically, the digital overlay application 101 can establish orinterface with an interactive video telecommunications session, forexample, over a public network 1050, with the digital overlayapplication's datacenter 130, which can provide for real-time dataexchange interactions with a plurality of third parties during an event200. In a networked deployment, the processing system may operate in thecapacity of a server 140 or a video telecommunications machine in avideo telecommunications-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment.

Multiple users may collaborate or work separately on one event 200.Multiple users may work in tandem or separately from the same terminalor from different remote terminals. In some embodiments, each event 200may include one or more unique digital overlay screens 112 comprisingone or more fields 220 with desired digital content 240 associated withone or more unique user presentations or videos which may be rendered atsome later point in time. For example, two users in the exact samephysical location may create separate digital overlay screens 112 withfields 220 comprising different content 240 for the same event 200,which are associated with one presentation occurring at a future time.In this instance, when User 2 takes over the event 200 from User 1 atthe same terminal/user device 110, then the digital overlay application101 can be automatically or manually toggled, e.g., by pressing a key orcombination of keys, to User 2's digital overlay screen 112 and field220 associated with their personalized, predefined event content 240.The presenting user's digital overlay screen 112 is configured tooverlay over the underlying platform or program 100, and video feed 120.

Turning to FIG. 5, each user is able to manage their portion of theevent 200 and take control thereof by utilizing the application's 101event controls 170 which include one or more selectable options (e.g.,icons, buttons, etc.) relating to parameters defining an event 200, itsdigital overlay screen 112/content 240, and users associated therewith.For example, a user may switch profiles by using the event controls 170,and by doing so, take control of a given event 200 by generating theirdigital overlay screen 112 on the displayed user interface 115. Here,for example, the event controls 170 change the user interface 115 bydisplaying the digital overlay screen 112 of a different user for thesame event 200.

The user interface 115 may comprise a title bar 510 with a visualindication 500 (e.g., an icon) that indicates other user profilesavailable for presentation in the event 200. In some embodiments,clicking on the visual indication 500 allows the user to browse andclick on an element in order to select a new user to take control ofsaid event 200. As illustrated in FIG. 5, the user interface 115 mayalso include a dropdown script selection element 520 which displays asorted list of identifying information associated with a user's profile,e.g., a user's log-in handle (@User1, @User 2, @User 3, etc.). The usermay click on the information associated with a specific user's profile,or enter identifying text corresponding to the specific user's profile(i.e., their user name), so as to make them the controlling user forthat event 200. In some embodiments, when multiple digital overlayscreens 112 or fields 220 are to be used for a single event 200, usersmay pre-fix the order in which the overlay screens 112 are deployedduring the event 200. In some embodiments, the dropdown script selectionelement 520 provides a “next user” button, as shown in FIG. 5, whichallows the digital overlay screens 112 to be dynamically populated inthe pre-set order determined by the user(s)/event administrator(s).Although the event controls 170 described above depict interfaceelements for rendering control of display overlay screens 112, oneskilled in the art will recognize that other types of interface elementsmay be used.

FIG. 6 illustrates an exemplary process 600 for the accessibilitycontrols 610 of an event. In some embodiments, users may restrict orlimit accessibility of an event 200 by using accessibility controls 610.Specifically, a user may restrict who may attend or limit access to aparticular event 200, and also grant non-presenter associates theability to create and “push” content boxes to the digital overlay screen112 of the user, presenting in real-time during the rendering of anevent 200.

When creating an event 200, a user may use accessibility controls toinvite associates (block 601). Each user and invited associate must haveunique application 101 credentials to participate. Associates invited toan event 200 are granted access to view content 240 therein. However,editing controls may limit or restrict an invited associates' access(block 602). Specifically, an invited associate not granted editingrights will only be allowed to view the event 200 (block 603).Alternatively, an invited associate granted editing rights for an event200 will be given the option to view and/or edit the content 240 (block604). Further, the user may limit the invited associate's editing rights(block 605). An associate with limited editing rights may modify content240 pursuant to the limitations established by the user (block 606). Anassociate granted editing rights not limited in scope, may modify allcontent 240, without limitation, and in real-time, throughout the entireduration of the event 200 (block 607). In some embodiments, the user maychoose to lock editing rights for particular fields 220 and leave otherfields 220 modifiable throughout the duration of an event 200.

Furthermore, users may choose parameters which require a confirmationrequest prior to content modifications or notes transmitted from thirdparties being applied to the field 200. Based on the user's response tothe confirmation request, the application 101 may modify the content 240of the event 200. In this way, the user can confirm or decline anyautomatic or transmitted modifications or additions to content 240. Theuser can confirm using a tapping or clicking response to confirm saidchange. For instance, a single tap is to confirm the change and a doubletap is to decline the change.

In some embodiments, the user may choose to limit access to associates,so that a particular associate, for example, may only be given theoption to send chat messages, as described in further detail below,rather than push content 240 in fields 220 to the presenter's digitaloverlay screen 112. Other third-party event attendees may connect to thevideo feed 120 of a particular event 200 through the underlying networkconferencing platform 100, or the like. For example, a third-partyattendee may connect to the video feed 120 for a particular event 200via Zoom, Facetime, etc. In this way, third party attendees connected tothe video feed 120 view the user, or presenter, speaking to the imagecapturing device 103, or camera, but do not have access to, receive, orsee the user's digital overlay screen 112 or its content 240 therein.One primary benefit of this technology is to help enhance thepresentation experience and quality of a presenter by allowing thepresenter to reference notes or facilitate real-time guidance from thirdparties during a presentation, without notifying viewers of theforegoing or creating the perception that the presenter has lost eyecontact with the camera and/or audience. Users may utilize thistechnology during a live presentation or a pre-recorded presentation.This technology among other things helps achieve the appearance to theaudience that the presenter is in command of the presentation and hasdiligently rehearsed or is well-prepared for the speech, presentation,interview, argument, etc. and helps the presenter better present becausethe presenter has the comfort of knowing notes and data are available asneeded without having to look away or look flustered.

The Digital Overlay System and Process

The digital overlay application 101 of the event-based digital overlaysystem 104 comprises an event-selection interface which allows users tocreate one or more digital overlay events 200. Each digital overlayevent 200 is associated with one or more digital overlay screens 112,which may comprise one or more fields 220 which are digitally displayedto a user via a user interface 115 of a user device 110, such as acomputing device. The digital overlay system 104 can include frontendservices such as, but not limited to, graphical user interfaces (GUI),communication modules, and application program interfaces (APIs) thatenable the application 101 to connect to a user device 110. Theapplication 101 can also include backend services, such as, but notlimited to machine learning (ML) and/or artificial intelligence (AI)algorithms (including but not limited to algorithms pertaining to voiceand eye analysis) configured to analyze various types of data collectedfrom the application 101 to create optimized digital overlay events 200for the user. For example, AI algorithms may monitor the user's speechlogistics during an event 200 to determine what action, if any, shouldbe taken.

As illustrated in FIGS. 7 through 9, in an embodiment, the userinterface 115 may be a GUI (or special interface or graphical trigger)configured to be dynamically presented on an electronic display of theuser's device 110 to display, (1) the video feed 120 generated at leastin part by the image capturing device or optically active device 103,such as a camera device, which may be communicatively coupled to anunderlying video rendering program 100, (2) the digital overlay screen112 generated by the application 101 and comprising interactivefeatures, such as a script or messages.

When a user logs into the application 101, they may open and uploadnecessary content 240 to one or more fields 220 of one or more digitaloverlay screens 112 for an event 200. A field 220 may comprisepredefined date, notes and/or content content boxes or customizableboxes which can be altered in size, color, dimension, shape, location,and/or placement. For example, the fields 220 may comprise square boxes,rectangles, circles, shapes which may be uniform in dimension, or whichmay vary in dimension along different axes. Various sizes, dimensions,shapes, configurations, and constructions can be determined by thoseskilled in the art. Furthermore, fields 220 may be configured such thatthey change size during an event 200. As illustrated in FIGS. 8 and 9,for example, a particular field 220 may remain minimized or small insize until its content 240 is being utilized in the video presentation,wherein (by either user selection in the moment, by pre-defined voicerecognition-based commands or other pre-determined directions, forexample) it maximizes or enlarges in size (the change in field 220 scaleis represented by broken lines in FIGS. 8 and 9) and configures itselfon the user interface 115 in a manner as to yield a line of sight of auser corresponding to the field of view of the camera device (and mayalso collapse away based on similar directions and/or commands). Fields220 may comprise content 240, such as, text, audio, images, documents,uploaded files, and the like, which are projected onto the userinterface 115 and accessible to the user during a rendering of an event200. In some embodiments, the user may generate handwritten notes asdigital content 240. For example, by utilizing a finger-stroke,stylus/smart pen, or device/method of the like, the user may write on atouch screen of the user device 100 (or other component of device 100compatible with said pen or input), and transmit handwritten notes ontothe device 100 such that they are displayed on the user interface 115 asdigital content 240. As illustrated in FIGS. 8 and 9, the user may inputhandwritten text that is digitized and generated on the field 220 ashandwritten digital content. In other embodiments, the application 101may include software known in the art which can recognize and convertinputted handwritten text into regular typeface such that the digitalcontent 240 produced on the user interface takes the form of digitaltypeface.

In an exemplar embodiment, the fields may include text of a script orfacts and data or other content which the user wishes to review orreference in real-time during the duration of an event 200. Content 240may be updated by users even during the duration of an event 200.Additionally, documents or images in, for example, Portable DocumentFormat (PDF), Microsoft Word (Word), Joint Photographic Experts Group(jpeg),and Portable Graphics Format (png), may be uploaded into a givenfield 220 and, if desired, uploaded into a pure text form, such that thecontent 240 may be viewable to the user during an event 200.

In one embodiment, the content 240 within a given field 220 isresponsive to shortcuts, such as Command-F (Ctrl-F), which allow thepresenter of an event 200 to quickly find words or phrases. In someembodiments, the digital overlay application 101 comprises a filesdatabase which stores script templates or pre-scripted content which theuser has saved and associated with their user profile and/or a givenproject, meeting, or event, and has the option to repopulate said scriptor template in preparation or use for a particular event 200.

As illustrated in FIG. 10, the digital overlay screen 112 may alsoinclude the chat interface referenced above. The application'sdatacenter 130 provides for real-time interaction with a plurality ofassociates. For example, an associate logged into the digital overlayapplication 101 and watching a particular event 200 may create a newcontent field or box 220 and push it to the presenter of said event 200,or deliver a message in real-time via the chat feature to the presenterbut which is not seen by the rest of the audience. The datacenter 130may include a plurality of server systems operating on a plurality ofserver machines communicatively coupled to each other via, for example,a public network and/or a secure virtual private network. The servermachines each include one or more processors, memory, an operatingsystem, one or more input/output interface and one or more networkinterfaces all known in the art.

Further, audio, pictures, files, multimedia, and other data andinformation may also be exchanged in real-time with the chat engine. Inone embodiment, the chat interface is comprised within a chat field orbox 900. In some embodiments, the chat field or pushed box or field 220comprising new content 240 will populate in a pre-determined location orin a different color than other boxes or fields 220 in the digitaloverlay screen 112 so as to alert, prompt, or notify the presenter thata new incoming message or content 240 has been received. Said alert,prompt or notification may be presented in an alert box 950. In someembodiments, a user is given the option to reserve a particular locationon the digital overlay screen 112 for real-time content boxes 220 thatare pushed to the main user or for messages exchanged through the chatinterface. For example, as illustrated in FIG. 10, the upper rightcorner of a user's digital overlay screen 112 may be reserved for pushedcontent from associates, so that the incoming content 240 appears in afield 220 on the upper right corner and may be opened and viewed by theuser. This allows for dynamic real-time updates to a specified field 220so that said fields 220 are not only static based on pre-set texts,data, content, information or notations. In some embodiments, content240 rendered within the fields 220 may be updated dynamically and inreal-time based on received instructions and/or metadata associated withthe incoming messages and push boxes 220. In some embodiments, the usermay also enable the projection of said messages and push boxes 220 to bedisplayed automatically as default, not to be displayed as a default, orto be displayed only upon the user opening the content.

Moreover, the fields 220 and chat box/fields 900 are only visible to thepresenter and not the viewers of the video presentation. If a videorecording is taking place, the chat fields 900 are not rendered as partof the video recording. In some embodiments, the field(s) 220, includingthe chat messages exchanged in a given chat boxes 900, are projected ina near translucent or transparent manner such that they at leastpartially superimpose or overlay the underlying video feed 120 withoutdistorting the video data captured and, further, in a manner in whichonly the user can see the content 240 of the field(s) and/or boxes 220.The digital overlay application 101 is configured to default field(s)220, font type, font size and color, and other parameters that mayimpact the display and appearance of the digital overlay screen 112,such that the content 240 displayed on the digital overlay screen 112 isvisible to the presenter and not visible to the viewers of the videofeed 120, and such that the user may maintain a line of sight with thecamera device 103, or camera device 103 communicatively coupled with theunderlying program 100. In some embodiments, the digital overlayapplication is concurrently running with the underlying networkconferencing program 100. In some embodiments, the digital overlayscreen 112 is configured such that its field(s) 220 and its controlssupplant over the controls of the underlying recording device or networkconferencing program 100. For example, a user can use a mouse or touchto navigate, open and close a field 220 in the application 101 andcontrol the scrolling functions thereof without manipulation of theimage capturing device's 103 or underlying running program's 103controls.

However, users may also have the option to customize font, font size,and font color of content 240 within the fields 220, including the sizeand shape of them, as they desire. Fields 220 may also be customized toappear on the display overlay screen 112 for a fixed or transitoryperiod, and/or, shift position, automatically/manually, either as afunction of a pre-set timer, clock, or in response to a cue recognizedby the application 101, such as a voice command.

The application 101 comprises an interface element comprising avoice-analyzing algorithm configured to determine and analyze speech, soas to adjust parameter controls in accordance with the user's speech orpresentation. Speed and text parameters of the fields 220 and content240 therein may enable the user to lock/unlock, open/close, orscroll/leave static the display of the fields/boxes 220, and content 240therein.

Moreover, different fields or boxes 220 may be configured with differentparameters. For example, a user may log into the digital overlayapplication 101, create an event 200 and upload one field 220 of content240 comprising scrolling functions and three fields 220 static inposition. Additionally, the speech and text parameters may be pre-set ata desired pace (like a teleprompter) so as to initiate the scrollingfunction for a field 220 comprising a script for a speech once the userbegins a speech or live presentation. The speech and text parameters mayalso be set to optionally respond to cues, instructions, and/or inputfrom the user as a control signal. Speech and text parameters may beconfigured to function at variable rates depending on the content 240and its construction, such as font and font size, and speed if manuallypre-determined. Specifically, the speed and text parameters may manuallyprompt the digital overlay application 101 to initiate a process changeon the user interface 115, though in other embodiments the changes maybe triggered without user input (i.e., automatically on behalf of theuser). More specifically, the speech and text parameters may stop thescrolling function of a particular field 220 based on a timer elapsing,an interaction with the user interface 115 indicating an instruction tostop scrolling, latency or no audio being detected, or the user goingoff script by way of recognizing the voice has departed from the pre-setspeech so the speech pauses. Audio, haptic, and/or visual feedback canalso be used to inform the user that the performance has departed fromthe pre-set speech rate. Visual guidance can be displayed to the userusing one or more colors, texts, images, or a combination thereof. Insome embodiments, the accessibility and control functions may becustomizable and set by the user. Note that the location of logic andstorage can be combined, separated, or even sub-divided in differentembodiments.

When a user has created the requisite content for an event 200, the userhas the option to set a desired start time so that the digital overlayscreen 112 opens automatically when the video event 200 is set to begin.A user may also indicate to the digital overlay application 101 when anevent 200 will occur so that its processor 109 is able to auto-open adigital overlay screen 112 at the set schedule of an event 200. The usermay also create the event 200 so that it opens at the time whenunderlying network conferencing platform or program 100 is opened. Inthis instance, the digital overlay screen 112 for the event 200 istethered so as to open over the video window of said platform or program100. Alternatively, the digital overlay application 101 can be openedmanually whenever the underlying platform 100 is opened and sized as theuser desires over the underlying platform 100.

Moreover, as illustrated in FIG. 10, the digital overlay screen 112 isconfigured to tether to the camera display of whatever optically activedevice 103, camera device, or network conferencing platform 100 is beingused so that the camera-default field of view has the digital overlayfield 220 over it. The field(s) 220 may be pre-populated so as to bedisplayed in a pre-set location on the field of view upon launch of theapplication 101 or underlying running program 100. Additionally oralternatively, the user may customize and save placement of the fields220 so as to arrange script and boxes 220 in a desired location on thedigital overlay screen 112 for display therein when an event occurs at alater time. In one embodiment, the digital overlay application 101 mayposition fields 220 on the digital overlay screen 112 so as to yield aline of sight of a user corresponding to the field of view of the cameradevice 103 or video program 100. In some embodiments, the digitaloverlay screen 112 is arranged such that it at least partially overlaysthe underlying video feed 120 and is positioned within the userinterface 115 such that the user appears to maintain eye contact with acamera 103, or the like. The displayed elements on the user interface115 may be arranged differently than the arrangements described or shownherein.

In another embodiment, the application 101 may recommend specificdigital overlay screen 112 and field 220 positioning a line of sight ofa user corresponding to the field of view of the tethered device orprogram 100 so as to assist the presenter in maintaining an optimizedviewpoint. For example, a script within a field 220 may be displayed ina position near the presenter's camera 103 so the speaker appears to bespeaking into the camera 103 as they read the field 220 of scriptedcontent 240 (e.g., speech, argument, presentation notes, etc.).

In yet another embodiment, the digital overlay application 101 maydynamically update the digital overlay screen 112 and arrangement of thefields 220 based on the user's current line of sight so that the userdoes not appear to divert their eyes away from the camera's 103 primaryline of sight. For example, the digital overlay application 101 maycomprise eye analyzing algorithms configured to monitor and detect eyegaze and movement, such that the position of the digital overlay screen112 and its fields 220 may be adjusted to compensate for shift in eyecontact so as to create the visual perception to the audience that theuser is looking directly into the camera 103, or at the audience itself,although they are reading from content 240 visually displayed on thedigital overlay screen 112. In an exemplar embodiment, the application101 interfaces with the optically active device's 103 (e.g., the camera)field of view such that if the eye positioning of a presenter appearsangled to the right, the digital overlay screen 112 may shift to theleft relative to its initial placement so that the user can reference orreview the field content 240 during an event 200 without appearing tohave disrupted eye contact with the audience (e.g., students, coworkers,a judge or jury, etc.). This allows the user or presenter to comfortablyreference notes in the fields 220 and/or communicate in real-time withthird parties through the chat interface or fields 220 during an event200, all while seeming or appearing to maintain eye contact with a mainfocal point of the tethered device or program 100. The digital overlayscreen 112 is configured such that the user may view the content 240 inthe fields 220 while concurrently viewing the video feed 120 andinteracting with the audience. The digital overlay screen 112 can betransparent and displayed simultaneously with the video feed 120. Inthis way, the digital overlay screen 112 does not significantly blockthe background image of the video feed 120. The video feed 120 maycomprise imagery data captured and outputted from the camera 103 of thepresenter's user device 110 during the rendering of a presentation.

Other eye detection, functionality, or AI algorithms determined by thoseskilled in the art may also be necessary for maintaining a central focalpoint with a tethered device or program 100. For example, a presenterutilizing AR glasses, wherein the digital AR overlay screen 112 isdisplayed thereon, may need to navigate the fields' content 240 usingeye tracking parameters specific to the AR glasses technology. In anexemplar embodiment, a user may be utilizing an optically active device103 which utilizes a gadget or neural implant, such as Neuralink. Inthis instance, functionality algorithms developed for the gadget may beintegrated with the digital overlay application 101 so as to allow theuser to maneuver content 240 in the digital overlay screen 112 displayedthereon in a manner native to said gadget or device. For example, a nearinvisible thimble device placed on a user's fingers, that when touched,may be configured to access and engage functions (e.g., play, stop, goleft, etc., similar to a computer mouse).

Both the digital event 200 itself and the content 240 thereof, includingbut not limited to the rendering of the video feed 120 and chat messagesexchanged in the fields 220, can be locally recorded (within a storagemodule of the application 101 or stored in an event-files database), ina remote data storage that is accessible to the digital overlayapplication 101, or a combination thereof. This enables, among otherthings, replay and history analysis by not only the user but alsoassociates so as to better refine and revise content 240 as needed. Thisalso allows the user to rehearse an event 200 and finalize actualcontent 240 to be placed for use in a digital overlay screen 112.

Terminology

Brief definitions of terms, abbreviations, and phrases used throughoutthis application are given below.

Reference in this specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the disclosure. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment, nor are separate or alternative embodimentsmutually exclusive of other embodiments. Moreover, various features aredescribed that may be exhibited by some embodiments and not by others.Similarly, various requirements are described that may be requirementsfor some embodiments but not others.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof, means any connection or coupling,either direct or indirect, between two or more elements; the coupling orconnection between the elements can be physical, logical, or acombination thereof. For example, two devices may be coupled directly toone another, or via one or more intermediary channels or devices.Devices may also be coupled in such a way that information can be passedthere between, while not sharing any physical connection with oneanother. Additionally, the words “herein,” and words of similar import,when used in this application, shall refer to this application as awhole and not to any particular portions of this application. Where thecontext permits, words in the Detailed Description using the singular orplural number may also include the plural or singular numberrespectively. The word “or,” in reference to a list of two or moreitems, covers all of the following interpretations of the word: any ofthe items in the list, all of the items in the list, and any combinationof the items in the list.

If the specification states a component or feature “may,” “can,”“could,” or “might” be included or have a characteristic, thatparticular component or feature is not required to be included or havethe characteristic.

The terminology used in the Detailed Description is intended to beinterpreted in its broadest reasonable manner, even though it is beingused in conjunction with certain examples. The terms used in thisspecification generally have their ordinary meanings in the art, withinthe context of the disclosure, and in the specific context where eachterm is used. For convenience, certain terms may be highlighted, forexample using capitalization, italics, and/or quotation marks. The useof highlighting has no influence on the scope and meaning of a term; thescope and meaning of a term is the same, in the same context, whether ornot it is highlighted. It will be appreciated that the same element canbe described in more than one way.

Consequently, alternative language and synonyms may be used for some ofthe terms discussed herein. Although synonyms for certain terms may beprovided, special significance is not to be placed on whether or not aterm is elaborated or discussed herein. A recital of one or moresynonyms does not exclude the use of other synonyms. The use of examplesanywhere in this specification, including examples of any termsdiscussed herein, is illustrative only and is not intended to furtherlimit the scope and meaning of the disclosure or of any exemplifiedterm. Likewise, the disclosure is not limited to various embodimentsgiven in this specification.

It should also be noted that all features, elements, components,functions, and steps described with respect to any embodiment providedherein are intended to be freely combinable and substitutable with thosefrom any other embodiment. If a certain feature, element, component,function, or step is described with respect to only one embodiment, thenit should be understood that that feature, element, component, function,or step can be used with every other embodiment described herein unlessexplicitly stated otherwise. This paragraph therefore serves asantecedent basis and written support for the introduction of claims, atany time, that combine features, elements, components, functions, andsteps from different embodiments, or that substitute features, elements,components, functions, and steps from one embodiment with those ofanother, even if the following description does not explicitly state, ina particular instance, that such combinations or substitutions arepossible. It is explicitly acknowledged that express recitation of everypossible combination and substitution is overly burdensome, especiallygiven that the permissibility of each and every such combination andsubstitution will be readily recognized by those of ordinary skill inthe art.

To the extent the embodiments disclosed herein include or operate inassociation with memory, storage, and/or computer readable media, thenthat memory, storage, and/or computer readable media are non-transitory.Accordingly, to the extent that memory, storage, and/or computerreadable media are covered by one or more claims, then that memory,storage, and/or computer readable media is only non-transitory.

When the embodiments are susceptible to various modifications andalternative forms, specific examples thereof have been shown in thedrawings and are herein described in detail. It should be understood,however, that these embodiments are not to be limited to the particularform disclosed, but to the contrary, these embodiments are to cover allmodifications, equivalents, and alternatives falling within the spiritof the disclosure. Furthermore, any features, functions, steps, orelements of the embodiments may be recited in or added to the claims, aswell as negative limitations that define the inventive scope of theclaims by features, functions, steps, or elements that are not withinthat scope.

It is to be understood that this disclosure is not limited to theparticular embodiments described herein, as such may, of course, vary.It is also to be understood that the terminology used herein is for thepurpose of describing particular embodiments only and is not intended tobe limiting.

As used herein and in the appended claims, the singular forms “a,” “an,”and “the” include plural referents unless the context clearly dictatesotherwise.

As used herein, the term “and/or” placed between a first entity and asecond entity means one of (1) the first entity, (2) the second entity,and (3) the first entity and the second entity. Multiple entities listedwith “and/or” should be construed in the same manner, i.e., “one ormore” of the entities so conjoined. Other entities may optionally bepresent other than the entities specifically identified by the “and/or”clause, whether related or unrelated to those entities specificallyidentified. Thus, as a non-limiting example, a reference to “A and/orB”, when used in conjunction with open-ended language such as“comprising” can refer, in one embodiment, to A only (optionallyincluding entities other than B); in another embodiment, to B only(optionally including entities other than A); in yet another embodiment,to both A and B (optionally including other entities). These entitiesmay refer to elements, actions, structures, steps, operations, values,and the like.

Terminology

In the foregoing specification, the subject matter has been describedwith reference to specific embodiments thereof. It will, however, beevident that various modifications and changes may be made theretowithout departing from the broader spirit and scope of the subjectmatter. For example, the reader is to understand that the specificordering and combination of process actions described herein is merelyillustrative, and the subject matter may appropriately be performedusing different or additional process actions, or a differentcombination or ordering of process actions. For example, this subjectmatter is particularly suited for video conferencing systems; however,the subject matter can be used for any optically active system ingeneral. Additionally and obviously, features may be added or subtractedas desired. Accordingly, the subject matter is not to be restrictedexcept in light of the attached claims and their equivalents.

1. A digital overlay system comprising: a user device comprising amemory, at least one processor configured to execute programs based onsoftware or a set of instructions from the memory, and an electronicdisplay, the user device configured to dynamically generate a userinterface to be presented on the electronic display; an image capturingdevice configured to capture a video feed in real-time, wherein thevideo feed comprises digital imagery data and is configured to betransmitted from the image capturing device and displayed on the userinterface and one or more remote devices; an application configured tobe stored and executed on the user device, wherein the at least oneprocessor is configured to initiate communication with the applicationto provide data to and request data from the application, wherein theapplication is further configured to interface with the image capturingdevice, and wherein the application is further configured to generate adigital overlay screen; and the digital overlay screen comprising one ormore fields having digital content, wherein the digital content isgenerated based on one or more inputs, wherein the one or more fieldsare configured to be projected on the digital overlay screen, whereinthe digital overlay screen is further configured to be dynamicallydisplayed on the user interface and at least partially superimposed onthe video feed displayed thereon, wherein the digital overlay screen isconfigured to not be transmitted to the one or more remote devices, andwherein the digital overlay screen is configured to tether to a cameradisplay of the image capturing device to create an appearance that auser is maintaining eye contact with the camera display whenconcurrently reading or viewing the digital content.
 2. The digitaloverlay system of claim 1, wherein the one or more remote devices areconnected to a network conferencing program, and wherein the applicationis further configured to interface with the network conferencingprogram.
 3. The digital overlay system of claim 2, wherein the networkconferencing program includes one of at least Zoom, Microsoft Teams,BlueJeans, FaceTime, Skype, Webex Meetings, or GoTo Meeting or othersimilar network video conferencing programs.
 4. The digital overlaysystem of claim 1, wherein the image capturing device comprises at leastone of a camera device of a personal computer, portable, mobile,wearable, or hand-held device.
 5. The digital overlay system of claim 1,wherein the application is further configured to provide log-incredentials to each user of the application, wherein the log-incredentials are configured to distinguish one user profile from anotheruser profile on the application, and wherein the application isconfigured to associate a respective digital overlay screen and thedigital content therein with a respective user profile.
 6. The digitaloverlay system of claim 1, wherein the digital overlay screen, thedigital content, and the video feed are locally recorded or stored in aremote data storage that is accessible to the application.
 7. Thedigital overlay system of claim, wherein the user interface includes atleast one of a graphical user interface, a special interface, or agraphical trigger.
 8. The digital overlay system of claim 1, wherein thedigital overlay system further comprises backend services, such as,machine learning and artificial intelligence algorithms configured toanalyze various types of data collected from the application.
 9. Thedigital overlay system of claim 1, wherein the digital content comprisesinteractive features, and wherein the digital content includes text,audio, images, documents, and uploaded files.
 10. The digital overlaysystem of claim 1, wherein the one or more fields comprise documents orimages in Portable Document Format, Microsoft Word format, JointPhotographic Experts Group, and Portable Graphics Format.
 11. Thedigital overlay system of claim 1, wherein the digital content isconfigured to respond to one or more shortcuts on the user device. 12.The digital overlay system of claim 11, wherein the one or moreshortcuts are configured to enable the user to search and find words andphrases in the one or more fields.
 13. The digital overlay system ofclaim 1, wherein the digital overlay screen is further configured toreceive an incoming box comprising new digital content from one or moreassociates of the user in real-time during the video feed.
 14. Thedigital overlay system of claim 13, wherein the incoming box isconfigured to populate on the digital overlay screen in a pre-determinedlocation.
 15. The digital overlay system of claim 13, wherein thedigital overlay screen further comprises an alert, prompt, ornotification when an incoming box is transmitted to the digital overlayscreen.
 16. The digital overlay system of claim 1, wherein the digitaloverlay screen further comprises or includes a chat interface, whereinthe chat interface is configured to enable the user to receive and/orexchange messages in real-time with one or more associates.
 17. Thedigital overlay system of claim 1, wherein the one or more fields areconfigured to dynamically update in real-time based on receivedinstructions associated with an incoming box comprising new digitalcontent from a remote device.
 18. The digital overlay system of claim 1,wherein the one or more fields and the digital content are neartranslucent or transparent, and wherein the one or more fields and thedigital content appear on the digital overlay screen without distortingthe underlying video feed.
 19. The digital overlay system of claim 1,wherein the image capturing device is communicatively coupled to anetwork conferencing program, wherein the application is furtherconfigured to concurrently run with the network conferencing program,and wherein one or more controls of the digital overlay screen areconfigured to supplant over the one or more controls of the networkconferencing program.
 20. The digital overlay system of claim 1, whereinthe font, font size, and font color of the digital content areconfigured to be customizable. 21.-32. (canceled)